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ABSTRACT 

The Aircraft Instrument Comprehension (AIC) Program 
is a self* instructional program designed to teach undergraduate 
student pilots to read instruments that indicate the position of the 
aircraft in flight, based on sequential- instructional stages of 
information, prompted practice, and unprompted practice. The program 
includes a 36«>item multiple choice test designed to assess the 
students' ability to read a compass and an artificial horizon. Three 
developmental tryouts among Air Force Reserve Officer Training Corps, 
students refined the test and the program. The first was conductea: 
vith three students vho vorked with the materials individually in a 
small observation room, and on the basis of this tryout revisions 
vere made to control the amount of exposure to the task prior to 
instruction. The second tryout of the program vas conducted vith 25 
students vho vorked through the materials in a large classroom; on 
the basis of this tryout the program vas revised to include 
additional practice items, and a pretest vas developed to assess 
student improvement as a result of instruction. The final tryout vas 
conducted among 32 students vorking in a large classroom, and both 
the pretest and the AIC test vere found to be acceptable. (JB) 
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DESIGN AND DEVELOPMENT OF THE 
AIRCRAFT INSTRUMENT COMPREHENSION PROGRAM 

Norman C. Higgins 

Educational research frequently fails to answer applied questions 
related to the design of effective instructional programs. Apparently 
contradictory results obtained from different studies of a particular 
instructional variable can often be attributed to the instructional in- 
adequacies of the materials and procedures used. The research literature 
on feedback, for example, is replete with studies in which the materials 
and procedures used contained such instructional inadequacies as insuffi- 
cient instruction, insufficient or inappropriate practice, and lack of 
an incentive for improved student performance. Research findings that 
result from the use of ineffective instructional materials are of little 
value as the basis for the systematic design of instruction. 

Tlie research and development activities that will be described 
in this symposium were undertaken to determine the effects of four 
potentially powerful instructional variables on undergraduate student 
pilot performance in ground training systems. The Aircraft Instrument 
Comprehension Program (Higgins, 1973) was systematically designed to 
facilitate the study of instructional cues, practice, feedback, and 
incentives. The program was tried out and revised and' its effective- 
ness was verified under classroom instructional conditions. The design 
and development of the program are described in this paper. The studies 
that were conducted to determine how selected variables contributed to the 
effectiveness of the program are described in two papers that follow. 
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The implications of this research and development activity for future 
research on instructional variables are described in the fourth and 
final paper of the symposium. 

Design of the AIC Program and Performance Measures 
The Aircraft Instrument Com prehension (AIC) Program is a self- 
ins true tionals^program designed to teach undergraduate student pilots 
to read instruments that indicate the position of an aircraft in flight. 
The content of the program was selected by training researchers in the 
Human Resources Laboratory, Williams AFB, Arizona, because it repre- 
sents a miati-dimensional concept similar in complexity to other content 
taught in USAF ground training programs. 

The AIC task requires a student to identify which one of four illus- 
trations of an aircraft in flight is most nearly in the position indicated 
by c compass a**u aiLiliciai horxzon. Inere are three dimensions to 
the position of an aircraft in flight: its heading, its pitch, and its 
bank. A compass is used to determine the heading of an aircraft. There 
are eight headings used in the AIC materials: four primary points 
(N, E, S, W) and four secondary points (NE, NW, SE, SW) . An artificial 
horizon is used to determine the pitch and the bank of an aircraft. 
There are three levels of pitch (30^ pitch-up, 0^ pitch, 30^ pitch-down) 
and three levels of bank (30^ bank- left, 0^ bank, 30° bank-right) used 
in the materials. Figure 1 illustrates the aircraft instruments and 
four illustrations of aircraft in flight. The aircraft labeled D is 
most nearly in the position indicated on the compass and artificial 
horizon in this item. 



The program is designed to successively approximate conditions 
under which the student is expected to identify an aircraft in any 
one of the 72 positions that can be derived from the dimension^ taught. 
The program is divided into four instructional phases that systemati- 
cally increase in their complexity. In the first instructional phase, 
the compass is presented alone and the student is taught to identify 
the heading of aircraft flying with 0"* pitch and 0"* bank. In the 
second phase, the artificial horis^.on is combined with the compass and 
the student is taught to identify the heading and the pitch of an 
aircraft flying with 0** bank. In the third phase, the student is 
taught to identify the heading and the bank of aircraft flying with 
0^ pitch. In the fourth and final phase of the program, the student 
practices identifying aircraft that vary on all three positional dimen- 
sions. 

Instruction within each of the first three phases of the program 
consists of (1) providing the information needed to read an instrument 
and providing examples of aircraft in positions indicated by the instru- 
ment; (2) providing prompted practice in reading the instruments; and 
(3) providing unprompted practice in reading the instruments* Each 
of the three instructional components (information, prompted practice, 
and unprompted practice) is presented on a separate page in the program. 

The program is bound into a loose-leaf binder to facilitate 
revision and experimental manipulation of each of the instructional 
components. Student responses to the, program practice items are re- 
corded on a separate response form. Confirmation of the accuracy of 
each response is provided by chemicalrbased feedback built into the 
response form. 



The Aircraft Instrument Comprehension Test is a 36 item multiple 
choice test designed to assess the students' ability to read a com- 
pass and an artificial horizon. The 36 items are designed to be 
representative of the 72 possible aircraft positions defined in the 
content description (8 levels of heading x 3 levels of pitch x 3 
levels of bank). Sample test items are included in Figures 1 and 2. 

Ihe instrument settings used in the item stems were systematically 
selected to insure that all possible combination of primary and second- 
ary heading, pitch, and bank are represented. The alternative response 
choices were systematically selected to either minimize or maximize 
the differences between the alternatives. The items designed to mini- 
mize the differences between the alternatives have illustrations that 
vary on only one of the positional dimensions and are minimally different 
on that one dimension (see Figure 1). The items designed to maximize 
the differences between the alternatives have illustrations that vary 
on two of the positional diiaensions and are maximally different on 
those two dimensions (see Figure 2) . 

The test includes directions for recording responses, a sample 
Jtem, and the 36 test items bound into a loose-leaf booklet. Student 
responses are recorded on mark sense response sheets that are separate 
from the test booklets. 

Developmental Testing 
The AIC Test and the AIC Program materials were reviewed for con- 
tent accuracy and clarity of presentation by USAF training personnel. 
The program was revised on the basis of their commentary prior to 
developmental testing. The initial developmental tryout was conducted 
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with a small number of students working under laboratory conditions. 
Subsequent tryouts were conducted with larger numbers of students 
working in a group situation that more nearly approximates classroom 
Instructional conditions. All tryouts were conducted with under- 
graduate students enrolled in Air Force Reserve Officer Training Corps 
programs. A pretes t-instruction~posttest design was used in each of 
the tryouts. 

Developmental Tryout 1 . The initial tryout of the program was 
conducted with three students who worked with the materials indivi-- 
dually in a small observation room. The developers observed how the 
students worked with the materials and recorded the time required to 
complete the test and the instructional program. Student responses 
to the test items and the program practice items were recorded on 
mark sense response sheets. 

The mean time required to complete the test prior to instruction 
was 15.9 minutes, the mean number of correct responses was 31.0 of a 
possible 36 responses (86% correct). The students worked straight, 
through the program without making any errors on the program practice 
items. All three students required less time to complete the test 
after instruction and improved their performance scores. The mean num- 
ber of correct responses was 34.3 of a possible 36 responses (94% 
correct) on the test after instruction. 

On the basis of this tryout, the directions in the AIC test were 
revised to exclude instruction related to the sample item and the time 
spent on the test prior to instruction was limited to 3 minutes. These 
revisions were made to control the amount of exposure to the task prior 
to instruction and to discourage students from trying to "discover" the 
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relationship between the instruments and the illustrations. No 
revisions were made in the program or in the procedures for adminis- 
tering the test after instruction. 

Developme ntal Tryout 2 . The second tryout of the program was 
conducted with a group of 25 students who worked through the materials 
in a large classroom. The students were directed to complete as many 
of the AIC Test items as possible in 3 minutes prior to being instructed 
on the task. Students worked at their own pace through the AIC Pro- 
gram. After instruction the students completed the AIC Test at their 
own pace. 

The mean number of items completed on the test prior to instruction 
was 8.8 items, the mean percentage of correct responses was 72%. TWelve 
of the twenty-five students scored 80% or higher on the test prior to 
instruction. The students worked straight through the program making 
very few errors. The mean number of correct responses on the program 
practice items was 13.1 of a possible 14 responses (94% correct). The 
mean number of correct responses on the test after instruction was 
31.7 of a possible 36 responses (88% correct). IWenty of the twenty- 
five students scored 80% or higher on the test after instruction. An 
analysis of the most frequently missed items on the test administered 
after instruction indicated the characteristics of the difficult 
items. The difficult items were of the type designed to minimize the 
difference between response choices and involved aircraft positions 
with secondary compass headings combined with pitched and/or banked 
attitudes . 

Based upon the results of this tryout, revisions were made in both 
the program and the test. The program was revised to include five 



additional practice items in the fourth phase of the program. The 
additional items had the characteristics of difficult items described 
above* A separate nine-item test was constucted for use prior to in- 
struction to identify those students who could demonstrate the greatest 
improvement in performance after instruction. The nine items were 
selected to represent the range of positions tested on the long 36 item 
test* The sequence of items on the longer version of the test was 
determined by random assignment; therefore, the first nine items were 
not representative of the possible positions. The procedures for adminis- 
tering the program and test materials were revised to include an incentive 
for correct responding on the test after instruction. 

Developmental Try out 3 . The third and final try out of the program 
was conducted with 32 students working in a large classroom. The 
students worked through the short, nine-item iorm of the test prior 
to instruction. Eleven of the thirty- two students scored 7 or higher 
on the test and were excused from further participation in the tryout. 
The 21 students who scored 6 or less on the test worked through the 
instructional program and the long form of the test at their own pace. 

The mean number of correct responses for the 21 students selected 
to participate in the tryout was 4.8 of a possible 9 responses (53% 
correct) on the test given prior to instruction* The mean number of 
correct responses on the program practice items was 16.6 of a possible 
19 responses (87% correct) . The mean number of correct responses on 
the test given after instruction was 32.0 of a possible 36 responses 
(87% correct). Eighteen of the twenty— one students scored 80% or 
higher on the test after instruction. 
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No revisions were made iu the program or test materials following 
the third tryout. The materials were considered effective enough to 
begin studies that would identify the contributions that each instruc- 
tional component made to student performance on the .aircraft instrument 
comprehension task. These studies and their* implications for future 
research will be described in the papers that follow. 
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